Traditional real-time object detection systems are “always on”, meaning they process every frame that is received from a video camera. However, running a real-time object detection model on a live video is computationally expensive and usually requires powerful hardware. Additionally, a traditional real-time object detection system does not display a video frame until it has been processed by the object detection algorithm. In less powerful hardware systems such as small computers that lack Graphic processing units (GPUs), this can result in a delayed output. The GPU is an electronic circuit specialized for parallel image processing and machine learning. Also, for typical surveillance use, most of the computation is wasted effort, because most views do not have objects of interest most of the time.
To address the above-mentioned problems, cloud computing is sometimes used. In a typical cloud based object detection system, the camera, smart phone, and local computer all transmit images to the server for cloud based recognition systems. However, cloud processing is slow and is not practical when real-time analysis is needed. For example, in a typical tactical video security system, real-time video information has to be made available to the end users on their mobile devices with a latency of less than one second. An isolated imaging device, such as a drone system that does not have a robust network connection, or a security camera that is not connected to a high-speed internet connection, may be referred to as edge devices. Non-real-time analysis is acceptable on cloud, but many dynamic, tactical, or security needs must have real-time analysis and require the complete processing to occur on an edge device. The major problem that edge devices have, as opposed to cloud video analysis systems, is that they lack processing power to run complex models (neural networks).
In view of the above, there is a need for an object detection system that ensures that the user always has a live camera view in critical scenarios such as security, and that requires less computation to process in real-time on CPU-limited edge devices. The object detection system must allow for smooth object-detection output on less powerful hardware such as small computers that lack GPUs, so as to save computational resources and electricity costs and therefore achieve longer operating time, especially on battery operated portable devices.